Score Calculator

The Score Calculator allows you to configure multiple score calculation strategies for optimizing search results within the Knowledge Base (KB). These strategies define how search queries are processed using a combination of keyword (Text) search, semantic (Vector) search, and an optional reranker to refine the results.

Matching Score Calculation

The matching score for a search result is determined using the following formula:

Matching Score = (Text Score × Text Weight) + (Vector Score × Vector Weight) + (Reranker Score × Reranker Weight)

Note:  The sum of Text Weight, Vector Weight, and Reranker Weight must always equal 100.

Adding a Score Calculator Strategy

The Knowledge Base (KB) comes with a default score calculator strategy, which balances keyword (Text) search and semantic (Vector) search. If needed, you can customize it or add new score calculator strategies to better fit your use case.

To create a new score calculator strategy, follow these steps:

  1. Navigate to Advanced Settings > Score Calculator section.
  2. Click on the Add new button.
  3. Set the score calculator strategy parameters:

    4. Parameter

    		 UI caption

    Description
    caption Caption The name of your strategy.

    topCount

    		 Top Count

    The total number of top matches to return.

    knnWeight

    		 Vector Weight

    The similarity score for semantic search. Decreasing this value may lead to more accurate results.

    bm25Weight

    		 Text Weight

    The similarity score for keyword search. Increasing this value may result in more noise in search results.

    rerankerWeight

    		 Reranker Weight

    The similarity score for the reranker. If you do not want to use the reranker, leave this value as "null".

    By default, the KB Engine utilizes the Orion reranker for bots created before version 8.13 and Cygnus for bots created in version 8.13 and later.

    You have the option to configure the reranker to HigherEducationReranker or leverage an LLM Reranker. Refer to the following sections for detailed instructions on setting these alternative rerankers.

    bm25TopCount

    		 Text Top Count

    The number of matches returned by the keyword (Text) search algorithm.

    knnTopCount

    		 Top Count

    The number of matches returned by the semantic (Vector) search algorithm.
    reranker Reranker

    Select the reranker you want to use for optimizing search results within the Knowledge Base.

    • Default behavior: For bots created before version 8.13, the Orion reranker is used by default. For bots created in version 8.13 and later, the default is Cygnus.

    • Higher education: Choose Higher Education to improve relevance when working with academic or educational content.

    • LLM-based reranking: Select Llm to use a large language model for reranking results. When selected, the Generative Endpoint field becomes available —here, you can choose from the generative endpoints defined in Advanced Settings > Generative Endpoints. The System and User prompts are pre-filled by default. We recommend not modifying these prompts unless you have advanced experience with prompt engineering.
  4. From the Active Score Calculator field, select the score calculator strategy to be used.

  5. Save the advanced settings.

To create a new score calculator strategy in DRUID version 8.13 and 8.14, follow these steps:

  1. Navigate to Advanced Settings > Advanced Settings JSON.
  2. Locate the scoreCalculator collection.
  3. Copy the structure of an existing score calculator strategy and paste it as a new entry.
  4. In the caption parameter, enter a name for the new strategy.
  5. Save your changes.
  6. Go to section Score Calculator and from the Active Score Calculator field, select the new strategy you created.
  7. Customize parameters to tailor how searches within the knowledge base are conducted, utilizing a combination of the keyword (Text) search algorithm and the semantic (Vector) search algorithm. Additionally, you can use the reranker to perform further analysis and enhance result quality.
  8. Save your changes.
Note:  Once configured, the Search Balance slider (found in KB General Settings) displays the matching score thresholds for the selected score calculator strategy.

Using the HigherEducation Reranker (in DRUID 8.13 and 8.14)

To optimize search results within your Knowledge Base (KB) containing higher education content, you can configure the HigherEducation reranker.

Navigate to KB Advanced Settings, click on Advanced Settings JSON, locate the "scoreCalculators" object, and set the "reranker" property to "HigherEducationReranker".

Using LLM Rerankers (in DRUID 8.13 and 8.14)

To optimize search results in the Knowledge Base (KB) using an LLM reranker in DRUID version 8.13 or 8.14, follow these steps:

  1. Go to KB Advanced Settings, and click Advanced Settings JSON.
  2. In the JSON editor, locate the "rerankers" array and find the LlmReranker object inside it.
  3. In the LlmReranker configuration, make sure to enter the name of the generative endpoint. This should match the caption you provided in the Generative Endpoints section of the UI (especially important if you renamed it from "Default").

  4. Locate the "scoreCalculators" object, and set the "reranker" property to "LlmKbReranker" or the name you have given to your LLM reranker.

  5. Save the advanced settings.